Papers with full re-training
CoME: An Unlearning-based Approach to Conflict-free Model Editing (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) often retain outdated or incorrect information from pre-training, which undermines their reliability. |
| Approach: | They propose a conflict-free model editing framework that selectively removes outdated knowledge from LLMs to improve their accuracy and reliability. |
| Outcome: | The proposed framework improves both editing accuracy and model reliability when applied to existing editing methods. |
Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Effective cross-lingual transfer is hindered by performance gaps and the scarcity of fine-tuning data in many languages. |
| Approach: | They propose a middle-layer alignment objective integrated into task-specific training to improve cross-lingual transfer across languages. |
| Outcome: | The proposed method improves cross-lingual transfer to lower-resource languages and can be merged with existing modules without full re-training. |
DEM: Distribution Edited Model for Training with Mixed Data Distributions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent fine-tuning approaches for large language models require supervised finetun on diverse datasets and follow different distributions. |
| Approach: | They propose a distribution edited model that integrates models individually trained on each data source with the base model using basic element-wise vector operations. |
| Outcome: | The proposed model outperforms baseline models on a variety of benchmarks and is cheaper than standard data mixing methods. |
MEAV: Model Editing with Alignment Vectors for inference time LLM alignment in single and multidomain preference spectrum (2026.findings-acl)
Copied to clipboard
Sadat Shahriar, Zheng Qi, Nikolaos Pappas, Srikanth Doss, Kishaloy Halder, Monica Sunkara, Manuel Mager, Yassine Benajiba
| Challenge: | Existing training-time alignment methods require full retraining when a change is needed. |
| Approach: | They propose an inference-time model-editing-based alignment method that learns encoded representations of preference dimensions and allows dynamic adjusting of the model behavior. |
| Outcome: | The proposed method can be used to align large language models to human preferences . it reduces the cost of inference by half compared to the prompt engineering approach . |